A detailed discussion of background information is set forth in U.S. patent application Ser. No. 108/22,445 and is incorporated by reference herein. Some more pertinent background is set forth below. All of the patents, patent applications, technical papers and other references referenced therein are incorporated herein by reference in their entirety. Various patents, patent applications, patent publications and other published documents are discussed below as background of the invention. No admission is made that any or all of these references are prior art and indeed, it is contemplated that they may not be available as prior art when interpreting 35 U.S.C. §102 in consideration of the claims of the present application.
1. Communication With Other Vehicles
The RtZF system of this invention can incorporate vehicle to vehicle communication allowing vehicles to inform other vehicles of their location, velocity, mass etc.
U.S. Pat. No. 05,506,584 to Boles relates to a system for communication between vehicles through a transmit and transponder relationship. The patent mentions that there may be as many as 90 vehicles within one half mile of an interrogation device in a multi-lane environment, where many of them may be at the same or nearly the same range. Boles utilizes a transponder device, the coded responses which are randomized in time, and an interrogation device which processes the return signals to provide vehicle identification, speed, location and transponder status information on vehicles to an operator or for storage in memory. No mention is made of how a vehicle knows its location or how accurate that knowledge is and therefore how it can transmit that location to other vehicles.
U.S. Pat. No. 05,128,669 to Dabbs provides for 2-way communication and addressing messages to specific vehicles. This is unnecessary and the communications can be general since the amount of information that is unique to one vehicle is small. A method of handing bi-directional communication is discussed in U.S. Pat. No. 05,506,584 to Boles. The preferred vehicle to vehicle communication system using pseudonoise techniques is more thoroughly discussed below.
In embodiments of the invention described herein, vehicle to vehicle communication is used, among other purposes, to allow the fact that one vehicle knows its position more accurately than another to use communication to cause the other vehicle to also improve the accuracy with which it knows its position.
2. Infrastructure to Vehicle Communication
The RtZF system of this invention can also incorporate communication from a vehicle to the infrastructure for a variety of reasons including obtaining the latest map updates, weather conditions, road conditions, speed limits, sign contents, accidents ahead, congestion ahead, construction, general internet access and for many other reasons.
The DGPS correction information can be broadcast over the radio data system (RDS) via FM transmitters for land use. A company called Differential Correction, Inc. has come up with a technique to transmit this DGPS information on the RDS channel. This technique has been used in Europe since 1994 and, in particular, Sweden has launched a nationwide DPGS service via the RDS (see, Sjoberg, Lars, “A ‘1 Meter’ Satellite Based Navigation Solutions for the Mobile Environment That Already Are Available Throughout Europe”). This system has the potential of providing accuracies on the premium service of between about 1 and 2 meters. A 1 meter accuracy, coupled with the carrier phase system to be described below, provides an accuracy substantially better than about 1 meter as preferred in the Road to Zero Fatalities™ (RtZF™) system of this invention.
In addition to the FM RDS system, the following other systems can be used to broadcast DGPS correction data: cellular mobile phones, satellite mobile phones, satellite internet, WiFi, WiMAX, MCA (multi-channel access), wireless tele-terminals, DARCs/RBDS (radio data systems/radio broadcast data system), type FM sub-carrier, exclusive wireless, and pagers. In particular, DARC type is used for vehicle information and communication systems so that its hardware can be shared. Alternately, the cellular phone system, coupled with the Internet, could be used for transmitting corrections (see, Ito, Toru and Nishiguchi, Hiroshi entitled “Development of DGPS using FM Sub-Carrier For ITS”). Primarily, as discussed elsewhere, vehicle to vehicle communications can be used to transmit DGPS corrections from one vehicle to another whether the source is a central DGPS system or one based on PPS or other system.
One approach for the cellular system is to use the GSM mobile telephone system, which is the Europe-wide standard. This can be used for transmitting DGPS and possibly map update information (see, Hob, A., Ilg, J. and Hampel, A. entitled “Integration Potential Of Traffic Telematics).
In Choi, Jong and Kim, Hoi, “An Interim Report: Building A Wireless Internet-Based Traveler's Information System As A Replacement Of Car Navigation Systems”, a system of showing congestion at intersections is broadcast to the vehicle through the Internet. The use of satellites is discussed as well as VCS system.
This is another example of the use of the Internet to provide highway users with up-to-date traffic congestion information. Nowhere in this example, however, is the Internet used to transmit map information. In fact, once there is an internet or equivalent connection to a vehicle then other information can be transmitted such as updated map information, weather and visibility, local conditions ahead, accident information, congestion information, DGPS corrections, etc. In fact, with a high bandwidth Internet connection, much of the computations, especially safety related computations, can best be done on the Internet where the system reliability would exceed that of a vehicle based system. The forecast that “the network is the computer” will begin to become reality. The crash of a safety related processor due to a software bug could not be tolerated in a safety related system and would be less likely to occur if the critical computations occur on the network. Furthermore, upgrades to vehicle based software also become feasible over such a high bandwidth connection.
A paper by Sheu, Dennis, Liaw, Jeff and Oshizawa, Al, entitled “A Communication System For In-Vehicle Navigation System” provides another description of the use of the Internet for real traffic information. However, the author (unnecessarily) complicates matters by using push technology which isn't absolutely necessary and with the belief that the Internet connection to a particular vehicle to allow all vehicles to communicate, would have to be stopped which, of course, is not the case. For example, consider the @home network where everyone on the network is connected all the time.
A paper by Rick Schuman entitled “Progress Towards Implementing Interoperable DSRC Systems In North America” describes the standards for dedicated short-range communications (DSRC). DSRC could be used for inter-vehicle communications, however, its range according to the ITS proposal to the Federal Government would be limited to about 90 meters although there have been recent proposals to extend this to about 1000 meters. Also, there may be a problem with interference from toll collection systems, etc. According to this reference, however, “it is likely that any widespread deployment of intersection collision avoidance or automated highways would utilize DSRC”. Ultra wide band communication systems, on the other hand, are a viable alternative to DSRC as explained below. The DSRC physical layer uses microwaves in the 902 to 928 megahertz band. However, ITS America submitted a petition to the FCC seeking to use the 5.85 to 5.925 gigahertz band for DSRC applications.
A version of CDPD, which is a commercially available mobile, wireless data network operated in the packet-switching mode, extends Internet protocol capabilities to cellular channels. This is reported on in a paper entitled “Intelligent Transportation Systems (ITS) Opportunity”.
According to a paper by Kelly, Robert, Povich, Doublas and Poole, Katherine entitled “Petition of Intelligent Transportation Society of America for Amendment Of The Commission's Rules to Add Intelligent Transportation Services (ITS) As A New Mobile Service With Co-Primary Status In The 5.850 to 5.925 GHz”, from 1989 to 1993 police received an annual average of over 6.25 million vehicle accident reports. During this same period, the total comprehensive cost to the nation of motor vehicle accidents exceeded the annual average of 400 billion dollars. In 1987 alone, Americans lost over 2 billion hours (approximately 22,800 years) sitting in traffic jams. Each driver in Washington D.C. wastes an average of 70 hours per year idling in traffic. From 1986 to 1996, car travel has increased almost 40% which amounts to about a 3.4% increase per year.
Further, from Kelly et al., the FCC has allocated in Docket 94-124, 46.7 to 46.9 GHz and 76 to 77 GHz bands for unlicensed vehicular collision avoidance radar. The petition for DSRC calls for a range of up to about 50 meters. This would not be sufficient for the RtZF™ system. For example, in the case of a car passing another car at 150 kilometers per hour. Fifty meters amounts to about one second, which would be insufficient time for the passing vehicle to complete the passing and return to the safe lane. Something more in the order of about 500 meters would be more appropriate. This, however, may interfere with other uses of DSRC such as automatic toll taking, etc., thus DSRC may not be the optimum communication system for communication between vehicles. DSRC is expected to operate at a data rate of approximately 600 kbps. DSRC is expected to use channels that are six megahertz wide. It might be possible to allocate one or more of the six megahertz channels to the RtZF™ system.
On DSRC Executive Roundtable—Meeting Summary, Appendix I—Proposed Changes to FCC Regulations covering the proposed changes to the FCC regulations, it is stated that “. . . DSRCS systems utilize non-voice radio techniques to transfer data over short distances between roadside and mobile units, between mobile units and between portable and mobile units to perform operations related to the improvement of traffic flow, traffic safety and other intelligent transportation service applications . . . ”, etc.
A state or the Federal Government may require in the future that all vehicles have passive transponders such as RFID tags. This could be part of the registration system for the vehicle and, in fact, could even be part of the license plate. This is somewhat discussed in a paper by Shladover, Steven entitled “Cooperative Advanced Vehicle Control and Safety Systems (AVCSS)”. AVCSS sensors will make it easy to detect the presence, location and identity of all other vehicles in their vicinity. Passive radio frequency transponders are discussed. The use of differential GPS with accuracies as good as about two (2) centimeters, coupled with an inertial guidance system, is discussed, as is the ability of vehicles to communicate their locations to other vehicles. It discusses the use of accurate maps, but not of lateral vehicle control using these maps. It is obvious from reading this paper that the author did not contemplate the safety system aspects of using accurate maps and accurate GPS. In fact, the author stresses the importance of cooperation between various government levels and agencies and the private sector in order to make AVCSS feasible. “Automotive suppliers cannot sell infrastructure-dependent systems to their customers until the very large majority of the infrastructure is suitable equipped.”
3. Limitations of the Prior Art
Previous inventions have attempted to solve the collision avoidance problem for each vehicle independently of the other vehicles on the roadway. Systems that predict vehicle trajectories generally fail because two vehicles can be on a collision course and within the last 0.1 second a slight change of direction avoids the collision. This is a common occurrence that depends on the actions of the individual drivers and no collision avoidance system now in existence is believed to be able to differentiate this case from an actual collision. In the present invention described below, every equipped vehicle will be confined to a corridor and to a position within that corridor where the corridor depends on sub-meter accurate digital maps. Only if that vehicle deviates from the corridor will an alarm sound or the vehicle control system take over control of the vehicle sufficiently to prevent the vehicle from leaving its corridor if an accident would result from the departure from that corridor.
Additionally, no prior art system is believed to have successfully used the GPS navigational system, or an augmented DGPS to locate a vehicle on a roadway with sufficient accuracy that that information can be used to prevent the equipped vehicle from leaving the roadway or striking another similarly equipped vehicle.
Prior art systems in addition to being poor at locating potential hazards on the roadway, have not been able to ascertain whether they are in fact on the roadway or off on the side, whether they are threatening vehicles, static signs, overpasses etc. In fact, no credible attempt to date has been made to identify or categorize objects which may impact the subject vehicle.
The RtZF™ system in accordance with this invention also contemplates a different kind of interrogating system. It is optionally based on scanning infrared laser radar, terahertz radar with or without range gating. This system, when used in conjunction with accurate maps, will permit a precise imaging of an object on the road in front of the vehicle, for example, permitting it to be identified (using neural networks) and its location, velocity and the probability of a collision to be determined.
In particular, the system of this invention is particularly effective in eliminating accidents at intersections caused by drivers running stop signs, red stoplights and turning into oncoming traffic. There are approximately one million such accidents and they are the largest killer in older drivers who frequently get confused at intersections.
4. Definitions
“Pattern recognition” as used herein will generally mean any system which processes a signal that is generated by an object (e.g., representative of a pattern of returned or received impulses, waves or other physical property specific to and/or characteristic of and/or representative of that object) or is modified by interacting with an object, in order to determine to which one of a set of classes that the object belongs. Such a system might determine only that the object is or is not a member of one specified class, or it might attempt to assign the object to one of a larger set of specified classes, or find that it is not a member of any of the classes in the set. The signals processed are generally a series of electrical signals coming from transducers that are sensitive to acoustic (ultrasonic) or electromagnetic radiation (e.g., visible light, infrared radiation, capacitance or electric and/or magnetic fields), although other sources of information are frequently included. Pattern recognition systems generally involve the creation of a set of rules that permit the pattern to be recognized. These rules can be created by fuzzy logic systems, statistical correlations, or through sensor fusion methodologies as well as by trained pattern recognition systems such as neural networks, combination neural networks, cellular neural networks or support vector machines.
“Neural network” as used herein, unless stated otherwise, will generally mean a single neural network, a combination neural network, a cellular neural network, a support vector machine or any combinations thereof. For the purposes herein, a “neural network” is defined to include all such learning systems including cellular neural networks, support vector machines and other kernel-based learning systems and methods, cellular automata and all other pattern recognition methods and systems that learn. A “combination neural network” as used herein will generally apply to any combination of two or more neural networks as most broadly defined that are either connected together or that analyze all or a portion of the input data.
A trainable or a trained pattern recognition system as used herein generally means a pattern recognition system that is taught to recognize various patterns constituted within the signals by subjecting the system to a variety of examples. The most successful such system is the neural network used either singly or as a combination of neural networks. Thus, to generate the pattern recognition algorithm, test data is first obtained which constitutes a plurality of sets of returned waves, or wave patterns, or other information radiated or obtained from an object (or from the space in which the object will be situated in the passenger compartment, i.e., the space above the seat) and an indication of the identify of that object. A number of different objects are tested to obtain the unique patterns from each object. As such, the algorithm is generated, and stored in a computer processor, and which can later be applied to provide the identity of an object based on the wave pattern being received during use by a receiver connected to the processor and other information. For the purposes here, the identity of an object sometimes applies to not only the object itself but also to its location and/or orientation and velocity in the vicinity of the vehicle. For example, a vehicle that is stopped but pointing at the side of the host vehicle is different from the same vehicle that is approaching at such a velocity as to impact the host vehicle. Not all pattern recognition systems are trained systems and not all trained systems are neural networks. Other pattern recognition systems are based on fuzzy logic, sensor fusion, Kalman filters, correlation as well as linear and non-linear regression. Still other pattern recognition systems are hybrids of more than one system such as neural-fuzzy systems.
The use of pattern recognition, or more particularly how it is used, is important to the instant invention. In the above-cited prior art, except in that assigned to the current assignee, pattern recognition which is based on training, as exemplified through the use of neural networks, is not mentioned for use in monitoring the interior passenger compartment or exterior environments of the vehicle in all of the aspects of the invention disclosed herein. Thus, the methods used to adapt such systems to a vehicle are also not mentioned.
A pattern recognition algorithm will thus generally mean an algorithm applying or obtained using any type of pattern recognition system, e.g., a neural network, sensor fusion, fuzzy logic, etc.
To “identify” as used herein will generally mean to determine that the object belongs to a particular set or class. The class may be one containing, for example, all motorcycles, one containing all trees, or all trees in the path of the host vehicle depending on the purpose of the system.
To “ascertain the identity of” as used herein with reference to an object will generally mean to determine the type or nature of the object (obtain information as to what the object is), i.e., that the object is an car, a car on a collision course with the host vehicle, a truck, a tree, a pedestrian, a deer etc.
A “rear seat” of a vehicle as used herein will generally mean any seat behind the front seat on which a driver sits. Thus, in minivans or other large vehicles where there are more than two rows of seats, each row of seats behind the driver is considered a rear seat and thus there may be more than one “rear seat” in such vehicles. The space behind the front seat includes any number of such rear seats as well as any trunk spaces or other rear areas such as are present in station wagons.
An “optical image” will generally mean any type of image obtained using electromagnetic radiation including visual, infrared, terahertz and radar radiation.
In the description herein on anticipatory sensing, the term “approaching” when used in connection with the mention of an object or vehicle approaching another will usually mean the relative motion of the object toward the vehicle having the anticipatory sensor system. Thus, in a side impact with a tree, the tree will be considered as approaching the side of the vehicle and impacting the vehicle. In other words, the coordinate system used in general will be a coordinate system residing in the target vehicle. The “target” vehicle is the vehicle that is being impacted. This convention permits a general description to cover all of the cases such as where (i) a moving vehicle impacts into the side of a stationary vehicle, (ii) where both vehicles are moving when they impact, or (iii) where a vehicle is moving sideways into a stationary vehicle, tree or wall.
“Vehicle” as used herein includes any container that is movable either under its own power or using power from another vehicle. It includes, but is not limited to, automobiles, trucks, railroad cars, ships, airplanes, trailers, shipping containers, barges, etc. The word “container” will frequently be used interchangeably with vehicle however a container will generally mean that part of a vehicle that separate from and in some cases may exist separately and away from the source of motive power. Thus a shipping container may exist in a shipping yard and a trailer may be parked in a parking lot without the tractor. The passenger compartment or a trunk of an automobile, on the other hand, are compartments of a container that generally only exists attaches to the vehicle chassis that also has an associated engine for moving the vehicle. Note a container can have one or a plurality of compartments.
“Out-of-position” as used for an occupant will generally mean that the occupant, either the driver or a passenger, is sufficiently close to an occupant protection apparatus (airbag) prior to deployment that he or she is likely to be more seriously injured by the deployment event itself than by the accident. It may also mean that the occupant is not positioned appropriately in order to attain the beneficial, restraining effects of the deployment of the airbag. As for the occupant being too close to the airbag, this typically occurs when the occupant's head or chest is closer than some distance such as about 5 inches from the deployment door of the airbag module. The actual distance where airbag deployment should be suppressed depends on the design of the airbag module and is typically farther for the passenger airbag than for the driver airbag.
“Transducer” or “transceiver” as used herein will generally mean the combination of a transmitter and a receiver. In come cases, the same device will serve both as the transmitter and receiver while in others two separate devices adjacent to each other will be used. In some cases, a transmitter is not used and in such cases transducer will mean only a receiver. Transducers include, for example, capacitive, inductive, ultrasonic, electromagnetic (antenna, CCD, CMOS arrays, laser, radar transmitter, terahertz transmitter and receiver, focal plane array, pin or avalanche diode, etc.), electric field, weight measuring or sensing devices. In some cases, a transducer will be a single pixel either acting alone, in a linear or an array of some other appropriate shape. In some cases, a transducer may comprise two parts such as the plates of a capacitor or the antennas of an electric field sensor. Sometimes, one antenna or plate will communicate with several other antennas or plates and thus for the purposes herein, a transducer will be broadly defined to refer, in most cases, to any one of the plates of a capacitor or antennas of a field sensor and in some other cases a pair of such plates or antennas will comprise a transducer as determined by the context in which the term is used.
“Adaptation” as used here will generally represent the method by which a particular occupant or vehicle or other object sensing system is designed and arranged for a particular vehicle model. It includes such things as the process by which the number, kind and location of various transducers is determined. For pattern recognition systems, it includes the process by which the pattern recognition system is designed and then taught or made to recognize the desired patterns. In this connection, it will usually include (1) the method of training when training is used, (2) the makeup of the databases used, testing and validating the particular system, or, in the case of a neural network, the particular network architecture chosen, (3) the process by which environmental influences are incorporated into the system, and (4) any process for determining the pre-processing of the data or the post processing of the results of the pattern recognition system. The above list is illustrative and not exhaustive. Basically, adaptation includes all of the steps that are undertaken to adapt transducers and other sources of information to a particular vehicle to create the system that accurately identifies and/or determines the location of an occupant or other object in a vehicle or in the environment around the vehicle.
A “morphological characteristic” will generally mean any measurable property of a human such as height, weight, leg or arm length, head diameter, skin color or pattern, blood vessel pattern, voice pattern, finger prints, iris patterns, etc.
A “wave sensor” or “wave transducer” is generally any device which senses either ultrasonic or electromagnetic waves. An electromagnetic wave sensor, for example, includes devices that sense any portion of the electromagnetic spectrum from ultraviolet down to a few hertz. The most commonly used kinds of electromagnetic wave sensors include CCD and CMOS arrays for sensing visible and/or infrared waves, millimeter wave and microwave radar, and capacitive or electric and/or magnetic field monitoring sensors that rely on the dielectric constant of the object occupying a space but also rely on the time variation of the field, expressed by waves as defined below, to determine a change in state.
A “CCD” will be defined to include all devices, including CMOS arrays, APS arrays, QWIP arrays or equivalent, artificial retinas and particularly HDRC arrays, which are capable of converting light frequencies, including infrared, visible and ultraviolet, into electrical signals. The particular CCD array used for many of the applications disclosed herein is implemented on a single chip that is less than two centimeters on a side. Data from the CCD array is digitized and sent serially to an electronic circuit (at times designated 120 herein) containing a microprocessor for analysis of the digitized data. In order to minimize the amount of data that needs to be stored, initial processing of the image data takes place as it is being received from the CCD array, as discussed in more detail above. In some cases, some image processing can take place on the chip such as described in the Kage et al. artificial retina article referenced above.
The “windshield header” as used herein includes the space above the front windshield including the first few inches of the roof.
An “occupant protection apparatus” is any device, apparatus, system or component which is actuatable or deployable or includes a component which is actuatable or deployable for the purpose of attempting to reduce injury to the occupant in the event of a crash, rollover or other potential injurious event involving a vehicle
Inertial measurement unit (IMU), inertial navigation system (INS) and inertial reference unit (IRU) will in general be used be used interchangeably to mean a device having a plurality of accelerometers and a plurality of gyroscopes generally within the same package. Usually such a device will contain 3 accelerometers and 3 gyroscopes. In some cases a distinction will be made whereby the INS relates to an IMU or an IRU plus additional sensors and software such as a GPS, speedometer, odometer or other sensors plus optimizing software which may be based on a Kalman filter.
A precise positioning system or PPS is a system based on some information, usually of a physical nature, in the infrastructure that determines the precise location of a vehicle independently of a GPS based system or the IMU. Such a system is employed as a vehicle is traveling and passes a particular location. A PPS can make use of various technologies including radar, laser radar, terahertz radar, RFID tags located in the infrastructure, MIR transmitters and receivers. Such locations are identified on a map database resident within the vehicle. In one case, for example, the map database contains data from a terahertz radar continuous scan of the environment to the side of a vehicle from a device located on a vehicle and pointed 45 degrees up relative to the horizontal plane. The map database contains the exact location of the vehicle that corresponds to the scan. Another vehicle can then determine its location by comparing its scan data with that stored with the map database and when there is a match, the vehicle knows its location. Of course many other technologies can be used to accomplish a similar result.